Method for distance-vector routing using adaptive publish-subscribe mechanisms

ABSTRACT

A distance-vector based routing protocol that integrates with adaptive publish-subscribe mechanisms by establishing routes to well-known controllers using distance-vector signaling.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/030,366, filed on Apr. 18, 2016, which is a U.S. nationalstage application filed under 35 U.S.C. § 371 of InternationalApplication No. PCT/US2014/060922, filed Oct. 16, 2014, which claims thebenefit of and priority to U.S. Provisional Application No. 61/891,455,filed on Oct. 16, 2013. The entire disclosures of all of the foregoingapplications are incorporated by reference herein.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under GrantSUB700155/SC20070363 awarded by the US Army Research Office. Thegovernment has certain rights in the invention.

FIELD OF THE INVENTION

The presently disclosed subject matter is directed towards routing incomputer networks. More particularly, the present invention relates tointegrated routing and discovery services in computer networks usingdistances and the application of this method in wireless ad hocnetworks.

BACKGROUND

Telecommunications are fundamental to the operation of modern society.The performance and cost advantages of modern telecommunications havelead to the widespread use of computer networks to transmit voice,image, and data around the world. Those computer networks enablebillions of people to easily, quickly, and at low cost communicate withone another, share information, and transfer data.

Computer networks are based on an agreed set of protocols that enablemessages to be transmitted from a source, passed along thetelecommunication network, and received at a destination. Messages haveat least two parts, the actual information that the sender wants totransfer to the destination and the routing information that controlswho receives the message. The agreed upon protocols enable a sender toobtain the address of the destination and the various “stations” of thenetwork to handle the message as it passes through the network to thedestination.

A computer network is a collection of nodes (terminals or stations) thatare interconnected by links (a connection between two nodes). Each nodehas a unique network address. Some nodes are directly connected to oneanother (neighbors) while others are connected through intermediatenodes. The nodes use the message routing information to route thesource's message through the links and nodes from the sender node to thedestination node. Each transfer of the message from one node to anotheris a hop.

As used herein a ‘network’ is an interactive system of computers thatoften includes peripherals, terminals, and databases that are connectedby communications lines. Such communication lines may be wired, fiberoptic, wireless, microwave, line of sight, repeaters or othercommunication paths. The term ‘node’ is used herein expansively. A nodeis a communication connection point, a communication end point, or acommunication distribution point. It is an active entity capable ofsending, receiving, or forwarding information over a communicationschannel. A physical node may be a computer, a terminal or a peripheral.A virtual node may exist as a code construct. While different networksmay use different types of nodes the term node should be understood asany entity that receives and transmits messages. By ‘dynamic’ it ismeant an interactive network or process is characterized by change andadaptation. A dynamic network adapts to changes and to inputs to andfrom itself as required to complete its tasks. For example, a new nodemay join a dynamic network. Required processes such as node addressesare kept and distributed as needed. Nodes can leave, and other networksmay be merged. All the while the network still services its clients.

A small ‘fixed’ computer network with stable members can relativelyeasily determine the correct or best path for passing messages betweenany two nodes. But as the network gets larger the number of ‘correct’paths from one node to the others gets very large. There are numerousmethods of selecting ‘correct’ paths. However, no matter the method itis beneficial to reduce the routing information signal handlingrequirements.

Greatly complicating the difficultly of choosing a correct path totransfer messages between a source node and a destination node arewireless ad hoc networks. A wireless ad hoc network is a wirelessnetwork that sets itself up external of a pre-existing infrastructure.Wireless ad hoc networks are decentralized in the sense that nocontrolling node manages the entire network. Each wireless ad hocnetwork node participates in routing by forwarding messages for othernodes dynamically based on network connectivity.

An ad hoc network should be understood broadly as an improvised,possibly impromptu, dynamically formed network. An ad hoc network formsas needed to service its clients. Then it grows, shrinks, moves, anddissipates as needed.

Traditional routing protocols for computer networks and wireless ad hocnetworks in particular rely on network-wide dissemination of signalingpackets that provide proactive updates to the state of links (whichnodes are connected by links or the distances to destinations), oron-demand requests for routes to destinations. However, as the number ofnodes, dynamic connectivity changes, and new traffic flows increase bothapproaches tend to incur excessive signaling overhead.

Making the problem of determining message paths even more difficult aremobile ad hoc networks (MANET). In a MANET a particular node may changeits physical location, which changes the nearest nodes it can directlyconnect to, which changes “best” message paths, which makes keepingtrack of good message paths even more difficult. The signaling overheadrequired to track all nodes in a network and to determine how to passmessages from any particular source node to any particular destinationnode can be prohibitive.

In a prior art computer network a particular node might be a serverhaving services to offer, a client in need of services, or a router forpassing information. To properly use a service a node must be made awareof the availability of that service. For example, to send a message asender node must be able to find the address of a destination node and amessage path. The sender node must use a service that can supply thatinformation, and for that the sender must know how to contact theservice. Publishing that information requires service discovery.Considerable work has been performed regarding implementing servicediscovery in computer networks. Typically service-discovery is performedby operating on top of a routing infrastructure or as an augment of anexisting routing protocol.

Interestingly, prior solutions for making routing more scalable do notintegrate destination-based routing with an adaptive publish-subscribemechanism in a way that reduces the signaling required for both routingand service discovery.

Prior routing protocols in computer networks and MANETs assumed that themappings of destination names to either addresses or routes wereindependent of actual routing. Various routing approaches were used,including hierarchical, limiting the dissemination of control messages,distributed hash tables (DHT), Bloom filters, virtual or geographicalcoordinates, or sets of dominating nodes to reduce the size of routingtables or the amount of route signaling.

Hierarchical routing schemes organize nodes into clusters. Somehierarchical routing schemes reduce signaling overhead by limiting thepropagation of control messages based on their distance from anoriginating node. One problem with hierarchical routing schemes in aMANET is that the affiliation of nodes to specific clusters is easilybroken when a node moves. Re-establishing affiliations incursconsiderable signaling overhead. Unless re-established, incorrectrouting can result in signaling decays based on distances to specificlinks

Distributed hash tables (DHT)-based schemes are attractive because adistributed hash table grows only logarithmically as the number ofdestinations grows. However, typical distributed hash table schemesdefine a virtual topology, which requires substantial signaling overheadto maintain the various links of virtual topologies that are defined inlarge MANET networks.

Automatic Incremental Routing (AIR) avoids virtual topologies by usingvariable-length prefix labels instead of addresses. Another approach isusing hashing node identifiers of destinations in Bloom filters, whichare then used in routing updates. Unfortunately, such schemes sufferfrom “false positives” that incur considerable signaling overhead.

Other routing schemes for MANET rely on geographical coordinates forrouting. Unfortunately such protocols require ubiquitous GPS servicesyet still incur routing signal overhead when discovering thegeo-locations of destinations node. Some other routing schemes usevirtual coordinates consisting of the distances between nodes andreference nodes. The main limitation of this type of routing scheme isthat the virtual coordinates of multiple nodes may be assigned the samevirtual coordinates. This is because there is simply no inherentuniqueness to a specific vector of distances to beacons. The result iseither possible incorrect routing or the use of additional signaling(typically flooding) to resolve false positives.

There are also many proposals in the prior art to reduce the number ofrelays needed to forward signaling messages for a given number ofdestinations. The best known example uses multipoint relays, forexample, P. Jacquet, P. Muhlethaler, T. Clausen, A. Laouiti, A. Qayyum,and L. Viennot, “Optimized Link State Routing Protocol for Ad HocNetworks,” IEEE INMIC 2001, pages 62-68, 2001. However, such proposalsrequire the establishment and maintenance of “connected dominatingsets,” i.e., the nodes selected to forward signaling messages must forma connected sub-graph. This usually requires a large subset of nodes,especially in dynamic topologies.

There has also been work performed on resource and service discovery inad hoc networks. Interestingly, such work either assumes that names aremapped to addresses and that routing to those addresses is independentof or augmented by existing routing protocols of service discoveryfunctionality.

In addition to the classic routing schemes discussed above, ad hocnetworks can also simply use flooding for forwarding data. A sendersends a message to its neighbors, which then pass that message on to itsother neighbors, and so on. This approach involves very large signalingoverhead, especially as the wireless ad hoc network grows.

In view of the foregoing a new approach to routing in computer networksbased on publish-subscribe mechanisms would be beneficial. Even morebeneficial would be an adaptive protocol for routing in wireless ad hocnetworks. In particular, an adaptive protocol that uses distance vectorsfor routing in computer networks and that integrates a sub-set of nodesto serve as controllers that maintain routes to nearby destinationnodes, maintains routes to all known controllers using distance vectors,and uses publish-subscribe mechanisms in which destinations informcontrollers of routes to them and in which sources obtain routes todestinations would be useful.

SUMMARY

The principles of the present invention provide for a novel approach torouting in computer networks, and MANETs in particular. The approachuses publish-subscribe mechanisms in an adaptive distance-vector routingprotocol. In particular, the new approach uses distance-vector protocolsfor routing and integrates a sub-set of nodes to serve as controllersthat maintain routes to nearby destination nodes, maintains routes toall known controllers using distance vectors, and uses publish-subscribemechanisms in which destinations inform controllers of routes to themand in which sources obtain routes to destinations.

The principles of the present invention are incorporated in a networkthat implements scalable integrated destination-based routing usingadaptive publish-subscribe mechanisms. First a subset of all nodes isselected to act as controllers that maintain routes to nearbydestinations. The routes to all known controllers are then maintainedusing distance vectors. Publish-subscribe mechanisms are thenimplemented in which destination nodes inform controllers of routes tothem and from which sources obtain routes to destinations. Beneficiallythe adaptive publish-subscribe mechanisms support routing to specificdestination nodes as well as copies of content that can be replicatedanywhere in the network.

The principles of the present invention further provide for a methodrouting that establishes a wireless network comprised of a plurality ofnodes, with each node having a unique node identifier (such as anaddress). Then, dynamically selecting a subset of those nodes to serveas controllers, thereby dividing the nodes into a plurality ofcontroller nodes and a plurality of destination nodes. Furthermore, thatmethod includes maintaining routes to nearby destination nodes of eachcontroller node, thereby forming local controllers, maintaining routesto all controllers in each controller based on distance vectors, andusing publish-subscribe mechanisms in which destinations inform localcontrollers of routes to them and sources obtain routes to localcontrollers of destination nodes.

BRIEF DESCRIPTION OF DRAWINGS

The advantages and features of the present invention will become betterunderstood with reference to the following detailed description andclaims when taken in conjunction with the accompanying drawings, inwhich like elements are identified with like symbols, and in which:

FIG. 1 is a topological view of a distributed network having links and avariety of nodes for implementing a method that is in accord with theprinciples of the present invention.

DETAILED DESCRIPTION

The presently disclosed subject matter now will be described more fullyhereinafter with reference to the accompanying FIGURES in which oneembodiment is shown. However, it should be understood that thisinvention may take different forms and thus the invention should not beconstrued as being limited to the specific embodiment set forth herein.

All documents and references referred to in this disclosure are herebyincorporated by reference for all purposes. Additionally, in the FIGURESlike numbers refer to like elements throughout. The terms “a” and “an”as used herein do not denote a limitation of quantity, but rather denotethe presence of at least one of the referenced items.

The present invention is described herein with reference to FIG. 1.Specifically, the present invention provides an AdaptivePublish-Subscribe Distance Vector (APDV) approach to maintaining acommunication network that is or that includes a network or sub-network.APDV assumes that each network node is assigned a network-wide uniquenode identifier. APDV can take advantage of the broadcast nature ofradio links by having each network node broadcast control messages toall neighbors once, rather than transmitting a separate control messageto each neighboring node.

In APDV a subset of nodes are dynamically selected to serve ascontrollers. Once selected a controller acts as a directory server forother nodes by maintaining routes to nearby destinations that aredenoted by node identifiers. This dynamic selection is based on adistributed algorithm that selects controllers to ensure that eachnon-controller destination node is within a maximum distance r from aminimum number k of local controllers. As a side effect a controllernode informs each destination node about routes to its one-hop andtwo-hop neighbors.

FIG. 1 illustrates a proto-typical computer network with APDV. FIG. 1 isuseful for illustrating and explaining the basic operation of APDV. InFIG. 1 each non-controller node has at least one local controller withintwo hops. The local controllers are nodes a, k, m, x and r. As isdescribed subsequently those nodes are dynamically elected to be thelocal controllers.

In APDV all nodes maintain routes to all controllers using a loop-freedistance-vector routing algorithm. For simplicity of explanation it willbe assumed that a node maintains a single route to each controller.However, in practice APDV can be readily extended to use multipleloop-free paths. Each non-controller node contacts each of its localcontrollers to publish its presence. That is accomplished by anon-controller node, for example non-controller node d sending a publishmessage to each of its local controllers with the mapping (d, {l¹ _(d) .. . l^(k) _(d)}), where l¹ _(d) (1≤i≤k) is a local controller for thenon-controller node d. Each local controller l^(i) _(d) of thenon-controller node d and each relay node (a node that passesinformation between other nodes) between the non-controller node d and alocal controller that receives the publish request stores a tuplestating the address of the non-controller node d, the next hop tonon-controller node d, and the address of the local controllers {l¹_(d), . . . l^(k) _(d)}.

In addition, the non-controller node d uses a common hash function toselect an anchor controller a_(d), which is a selected controller forstoring the mapping (d, {l¹ _(d), . . . l^(k) _(d)}). That mapping isthe addresses of all of the local controllers of non-controller node d.That mapping is sent to the anchor controller a_(d). The anchorcontroller a_(d) and the relay nodes between d and a_(d) cache themapping information. In FIG. 1, node d has published its presence withits local controller (node r) and its anchor controller (node a_(d)).

The foregoing produces the important result that each local controllernode r has a route to non-controller node d; the relay nodes betweeneach local controller node r and non-controller node d have the pathfrom local controller node r to non-controller node d; the anchorcontroller a_(d) has a route to each local controller node r; and therelay nodes between the anchor controller a_(d) and each localcontroller node r have the path between them.

A source node requiring a route to a non-controller node d uses the samecommon hash function on the identifier of node d to find the anchorcontroller for d, a_(d). Thus any source node can send a subscriptionrequest to anchor controller a_(d) by providing d and (s, {l¹ _(s), . .. l^(k) _(s)}), where l¹ _(s)(1≤i≤k) is a local controller for node s.The controller a_(d) responds with the mapping (d, {l¹ _(d), . . . l^(k)_(d)}) and sends that response towards the nearest local controller forsource s selected from the set {l¹ _(s), . . . , l^(k) _(s)}. The answeris redirected to source s by either the selected controller l^(j) _(s)or the first relay node along the route from a_(d) to controller k witha route to s.

Node s can then send data packets to destination d by sending themtowards the nearest controller in the set {l¹ _(d), . . . l^(k) _(d)}.Those packets will be redirected to d after either reaching the selectedcontroller in {l¹ _(d), . . . , l^(k) _(d)} or a node along the routefrom s to the selected controller with an active route to d. In FIG. 1,node s subscribes to node d by contacting anchor a_(d), which maintainsthe mapping (d, r) and returns it to node s by sending its responsetowards node k, which is the local controller of node s. Node a_(d) alsocaches the mapping (s, k). Node s then sends data packets to d bysending them towards controller r; however, node y is in the route froms to r and also has a route to d, and forwards the data packets directlyto d.

A proper implementation of APDV requires appropriate methods to selectcontrollers, to maintain the routes to the controllers, and thenpublishing and subscribing to destinations using the controllers.Furthermore, the information maintained at each node must allow thatnode to select and route to controllers, to route to local destinations,and to learn the local controllers associated with distant destinationson demand.

To achieve the foregoing a non-controller node i maintains a controllertable (CT^(i)) that states information about all controllers elected inthe network (the election process is described subsequently). Thenon-controller node i also maintains a neighbor controller table(NCT^(i)) stating information reported by each neighbor ofnon-controller node i regarding all controllers elected in the network;a neighbor table (NT^(i)) stating information about all one-hop andtwo-hop neighbors of non-controller node i; a neighbor local routingtable (NLRT^(i)) stating routing information reported by each neighborregarding all destinations within two hops and some destinations withinr hops; a local routing table (LRT^(i)) stating routing informationabout all destinations within two hops and some destinations within rhops; a neighbor routing table (NRT^(i)) stating information reported byeach neighbor regarding distant destinations; and a routing table (RV)stating information about distant destinations.

APDV employs a soft-state to operate efficiently.—A node transmits itsHELLOs periodically every 3 seconds. A HELLO includes some or all theupdates made to its node tables. A node stores all the information fromthe HELLOs it receives from its neighbors, and also caches informationit receives in subscription or publication requests from neighbors.Entries in RT^(i) and NRT^(i) are populated by the publish-subscribesignaling described subsequently. NT^(i), CT^(i), NCT^(i), LCL^(i),NLRT^(i), and LRT^(i) are updated by the exchange of HELLOs amongone-hop neighbors. In a network with point-to-point links, a nodetransmits a HELLO over each of the links it shares with its neighbornodes. In a network with broadcast links (e.g., a wireless network basedon broadcast links) a node transmits a HELLO once and addresses it toall its neighbors.

For each controller c selected in the network, CT^(i) specifies: theidentifier of node c (nid^(i) _(c)); the distance from i to c (d^(i)_(c)); the successors (next hops) from i to c (s^(i) _(c)); and asequence number (sn^(i) _(c)) used to avoid routing loops. N CT^(i)stores the controller tables reported by each neighbor of node i. Theentry for controller c reported by neighbor j and stored in NCT^(i) isdenoted by {nid^(i) _(cj), d^(i) _(cj), sn^(i) _(cj)}.

For each neighbor j of node i, NT^(i) specifies: the identifier of thenode (nid^(i) _(j)); a sequence number (sn^(i) _(j)) created by j andused to determine that the entry is the most recent from node J; acontroller status flag (cs^(i) _(j)) stating whether or not node j is acontroller; the controller counter (k^(i) _(j)) stating the number ofcontrollers within r hops of node j; and the local controller list(LCL^(i) _(j)) consisting of the identifiers of all controllers within rhops of node j. An entry for neighbor v in NT^(j) sent in a HELLO tonode i is denoted by {nid^(j)v, sn^(j)v, cs^(j)v, k^(j)v, LCL^(j)v}, andthe same entry stored in NT^(i) is denoted {nid^(i) _(vj), sn^(i) _(vj),cs^(i) _(vj), k^(i) _(vj), LCL^(i) _(vj)}.

An entry for destination j listed in LRT^(i) _(j)); the identifier ofthe node (nid^(i) _(j)); a sequence number (sn^(i) _(j)) created by jused to avoid routing loops; the distance from i to j (d^(i) _(j)); thesuccessor in the route to j (si^(j) _(i)); and the local controller listof node j (LCL^(i) _(j)), which may be a link to NT^(i) if the node iswithin two hops. An update made by neighbor j to LRT^(j) communicated ina HELLO is denoted by {nid^(j) _(v), sn^(j) _(v), d^(j) _(v), LCL^(j)_(v)}. The corresponding entry stored at node i in NLRT^(i) is denotedby {nid^(i) _(vj), sn^(i) _(vj), d^(i) _(vj), LCL^(i) _(vj)}. An entryfor destination v listed in RT^(i) simply specifies the identifier ofthe node (nid^(i) _(v)) and the list of local controllers for node v(LCL^(i) _(v)), because node i maintains the routes to all controllersin CT^(i).

Node i includes its own information in NT^(i), i.e., it stores an entrycorresponding to nid^(i) _(j), uses the information in its HELLOs. AHELLO from node i contains: nid^(i) _(i), sn^(i) _(i), cs^(i) _(i),k^(i) _(i), and updates to NT^(i), CT^(i) and LRT^(i). An update toNT^(i) regarding neighbor j consists of the tuple {nid^(i) _(j), sn^(i)_(j), cs^(i) _(j), k^(i) _(j), LCL^(i) _(j)}. An update to CT^(i)regarding controller c consists of the tuple {nid^(i) _(v), sn^(i) _(v),d^(i) _(v), LCL^(i) _(v)}.

With the foregoing information stored and exchanged within the variousnodes a distributed selection of controllers is required. In APDVselecting the controllers amounts to selecting a dominating set C ofnodes in the network to serve as controllers. Every node u that is not amember of C (called a simple node) is at a distance smaller than orequal to r hops from at least k nodes in C (called controllers). A nodeu is said to be (k, r) dominated (or covered) if there are at least knodes in C within r hops from u. There is a large body of work ondominating sets in graphs. Reference for example T. Haynes, S.Hedetniemi, and P. Slater “Fundamentals of Domination in Graphs,” MarcelDekker, 1998. Many other distributed algorithms exist to approximateminimum connected dominating sets MCDS with constraints.

However, the controller selection scheme used in APDV is simply aimed atobtaining a set of controllers that cover all nodes but it need not be aMCDS, and maintaining routes to all selected controllers. The APDVmethod is based on HELLO messages exchanged among one-hop neighbors. Tokeep the selection algorithm and signaling simple, only distances tocontrollers and node identifiers are used as the basis for the selectionof controllers.

In ADPV controllers self-select themselves to become controllers or tostop being controllers. A given node i determines whether to add ordelete its own entry in CT, respectively, according to the ControllerAddition Rules (CAR) presented below in Algorithm 1 and the ControllerDeletion Rules (CDR) presented below in Algorithm 2.

Algorithm 1 is represented as pseudo code for adding controllers to thetopology according to CAR:

#define DEFAULT_HELLO_INERVAL 3*SECOND #define DEFAULT_UPDATE_INERVAL20*SECOND Algorithm 1 Controller Addition Rule (CAR) Init: CT^(i)=φ_(i)and NT^(i)=φ if cs^(i) _(i) = = false & & k^(i) _(i) < k then minID =nid^(i) _(i); for ∀_(j) ∈ NT^(i) do if j ∉ CT^(i) && k^(i) _(j) < k thenminID = Min (minID, nid^(i) _(j)); end if end for if minID == nid^(i)_(i) then cs^(i) _(i) = true; k^(i) _(i)++; CT^(i) ← {nid^(i) _(i) = i,d^(i) _(i) = 0, sn^(i) _(i) = sn^(i) _(i) +1}; LCL^(i) _(i) ← {nid^(i)_(i) − i}; end if end if

Algorithm 2 is represented as pseudo code for deleting controllers fromthe topology according to CDR:

Algorithm 2 Controller Deletion Rule (CDR) If cs_(i) ^(i) == true &&k^(i) _(i) > k then CanDelete = =true; for ∀_(j) ∈ NT^(i) do if j ∉LCL^(i) _(i) && |LCL^(i) _(j) − {i}| < k then CanDelete = false; break;else if j ∈ LCL^(i) _(i) && nid^(i) _(i) < nid^(i) _(j) then CanDelete =false; break; end if if CanDelete == false then break; end if end for ifCanDelete == true then cs_(i) ^(i) = false; k_(i) ^(i) −−; CT^(i) ←{nid^(i) _(i) = i, d_(i) ^(i) = ∞ , sn^(i) _(i) = sn^(i) _(i) + 1}LCL^(i) _(i) ← LCL^(i) −{i}; end if end if

Node i is initialized with CT^(i)=φ and NT^(i)=φ, and waits for a fewseconds to start receiving HELLOs from nearby nodes. Hence, according tothe CAR, node i self selects itself as a controller when it is firstinitialized, unless node i has received HELLOs from neighbors thatprompt it not to include itself as a controller based on the CDR (seebelow). Node i updates an entry for j≠i∈CT^(i) according to the rulesshown in Algorithm 1. Those rules ensure that no loops are formed forroutes to controllers. Once node i has updated NT^(i) and CT^(i) byprocessing HELLOs received from neighbors, it computes its localcontroller list (LCL^(i)) from CT^(i), such that v∈LCL^(i) if div≤r, andsets k^(i) _(i)|LCL^(i)≡1.

The local controller list (LCL) at each node, and each new state pernode is determined by the reception of HELLOs from all neighbors,followed by the addition or deletion of controllers in the LCL resultingfrom applying CAR and CDR, thereby deleting controllers that are fartherthan 2 hops away, or deleting a controller after the successor to thatcontroller sends an update with a deletion of the controller.

Following the CAR each new node selects itself as a controller afterinitialization and sends a HELLO. Nodes do not wait for HELLOs toarrive. After receiving a HELLO from each neighbor, a node may add newcontrollers reported in the HELLOs. However, it also may delete itselffrom being a controller based on the CDR.

For simplicity of explanation it is assumed that each node maintains asingle route to each controller selected in the network using theupdates to controller tables included in HELLOs.

APDV uses a distance-vector routing approach to maintaining routes tocontrollers. To guarantee loop-free routes, APDV uses sequence numbersthat restrict the selection of next hops towards a given controller byany node, such that only those neighbors with shorter distances to thecontroller or with a more recent sequence number reported by thecontroller can be considered as successors. An important aspect of APDVis that entries for controllers can be deleted on purpose as a result ofthe CDR, rather than only as rare occurrences due to failures or networkpartitions. Together with the transmission of periodic HELLOs, the ResetController Rule (RCR) and the Update Controller Rule (UCR) discussedbelow address this functionality.

The process of routing to controller begins with the set N^(i); the setof one-hop neighbors of node i. Node i updates CT^(i) based on HELLOsfrom neighbor j∈N^(i) or the loss of connectivity to neighbor j. If nodei loses connectivity to node j, the entries in CT^(i) are deleted. Oncenode i is selected as a controller, it is the only node that can changethe sequence number for its own entry in controller-table updates sentin HELLOs.

When a given node i decides to delete itself as a controller based onthe CDR its entry must be deleted as a controller in the rest of thenetwork. To accomplish this node i uses a Reset Controller Rule (RCR)(provided below) to set its self-entry with an infinite distance and anup-to-date sequence number for a finite period of time T before deletingits self-entry from CT^(i). This ensures that the rest of the nodesdelete i from their controller tables.

If node i receives a HELLO from j or experiences a link failure thatmakes it update CT^(i) for entry c≠i: {nid^(i) _(cj), d^(i) _(cj),sn^(i) _(cj)}, node i updates its entry for c in CT^(i) according toUpdate Controller Rules (UCR), see Algorithm 3 below, which forces nodei to propagate a reset update or to select a successor to controller cthat is either closer to c or has reported a more recent sequence numberfrom c.

Algorithm 3 is represented as pseudo code for UCR used in the topology:

Algorithm 3 Update Controller Rule (UCR) Event Trigger: node i updatesentry for c in CT^(i) If 

 _(q) ∈ NT^(i) && sn^(i) _(cj) > sn^(i) _(c) then if n = = s^(i) _(c) && sn^(i) _(cj) > sn^(i) _(c) && d^(i) _(cv) == ∞ then sn^(i) _(c =)sn^(i) _(cv); d^(i) _(c=)∞;  else  maxSN = sn^(i) _(c);  for ∀_(v) ∈NT^(i) do maxSN = Max(maxSN_(i), sn^(i) _(c);)  end for  minD = d^(i)_(c) ; for ∀ f ∈ NT^(i) do if sn^(i) _(cf) == maxSN then if minD > d^(i)_(cj) + 1 then minD = d^(i) _(cj) + 1; temp_s = f; end if end if end ford^(i) _(c) = minD; s^(i) _(c) = temp_s; sn^(i) _(c) = maxSN; end if elseminD = d^(i) _(c); for ∀_(j) ∈ NT^(i) do if sn^(i) _(cj) == sn^(i) _(c)& & d^(i) _(cf) < d^(i) _(c) then if minD > d^(i) _(c) > d^(i) _(cf) + 1then minD = d^(i) _(cj) > d^(i) _(cf) + 1; temp_s = f; end if end if endfor d^(i) _(c) = minD; s^(i) _(c) = temp_s; end if

The RCR is:

RCR (Reset Controller Rule):

If node i must delete itself from CT^(i) using CDR then set d^(i)_(i)=∞; sn^(i) _(i)=sn^(i) _(i)+1; reset-timer^(i)=T

Nodes learn about routes to controllers and to one- and two-hopneighbors. However, exchanging HELLOs does not provide routes todestinations many hops away. To enable sources to obtain routes toarbitrary destinations without incurring network-wide dissemination ofsignaling messages, APDV uses a publish-subscribe mechanism forname-to-route resolution.

The basis of the publish-subscribe mechanism used in APDV is the use ofconsistent hashing. This is similar to recent proposals for distributedname resolution in MANETs that also use consistent hashing to map thenames of destinations to one of several predefined directory sitesstoring the name-to-address mapping for destinations. Key differencesexist however between the APDV approach and the prior art. Thosedifferences include: (a) the directories (i.e., controllers) areselected dynamically; (b) a node publishes its presence with multiplecontrollers; and (c) name resolution is integrated with the selection ofand routing to controllers, rather than running on top of routing.Hence, in APDV, controllers maintain name-to-route mappings rather thanstoring name-to-address mappings and then using an underlying routingprotocol to obtain the routes for known addresses.

For ease of explanation and understanding the APDV publish-subscribemechanism described herein assumes that node identifiers constitute thenames for which routes must be found. However, the same APDVpublish-subscribe mechanism can be used to support information-centricnetworking, such that nodes publish and subscribe to names ofdestinations, content or services, rather than just node identifiers.

Publishing in APDV consists of having a local controller know the routeto a given destination or having an anchor controller know the mappingfrom a node identifier to a list of local controllers. Subscribing inAPDV consists of a node requesting a way to reach a named destinationthrough an anchor controller.

In APDV, node i publishes itself with the k controllers listed inLCL^(i), and with one or more anchor controllers. The local controllersin LCL^(i) are within r hops of node i and serve as the “landmarks” forother nodes to submit data to node i, given that nodes far away fromnode i do not have routes to node i. Accordingly, a local controller fornode i must maintain a route to node i, and it also maintains themapping (i, LCL^(i)), so that it can find alternate ways to reach node iif its route to i fails. The anchor controllers are needed for nodes faraway from destinations to obtain the mappings between the identifiers ofthose destinations and their local controllers.

For simplicity, the assumption is that a single anchor controller isused for any one node. The anchor controller for node i (denoted bya_(i)) is obtained by using a network-wide consistent hash function thatmaps the identifier of node i into the identifier of one of thecontrollers selected in the network. Controller a_(i) must store themapping (i, LCL^(i)), so that it can provide any node v far away fromnode i the list LCL^(i), with which node v can send data packets towardsthe local controller in LCL^(i) that is nearest to node v according toits controller table CT^(v).

The forwarding of a publication request from a node to its localcontrollers is done by the exchange of HELLOs. Given that nodes maintainloop-free routes to all controllers, publication requests directed tolocal controllers of nodes are forwarded over the reverse loop-freeroutes already established from local controllers to nodes. The routesmaintained by local controllers to nearby nodes are refreshedperiodically; each node creates a new publication request by increasingthe sequence number included in the LRT self-entry of its own HELLO. Ifnode i receives a HELLO from neighbor j with a publication requestoriginated by node v, which consists of update to LRT^(j) fordestination v({nid^(j) _(v), sn^(j) _(v), d^(j) _(v), LCL^(j) _(v)})then node I forwards the request (i.e., it includes the LRT^(i) entry{nid^(i) _(v), sn^(i) _(v), d^(i) _(v), LDL^(i) _(x)} in its own HELLO)if it is the successor for node j to any of the controllers listed inLCL^(j) _(v). Once a local controller c receives an entry fordestination v and c∈LCL^(v), then c publishes (i.e. stores) the entry{nid^(c) _(v), sn^(c) _(v), d^(c) _(v), s^(c) _(v), LCL^(c) _(v)}, wheres^(c) _(v) is the neighbor from which it received the publicationrequest. Controller c may also forward it if it is the successor toanother controller in LCL^(v) for the neighbor from which it receivedthe publication request.

The submission of a publication request from node i to its anchorcontroller a_(i) is done by node i using the network-wide consistenthash function on the set of identifiers in CT^(i) to obtainhash(i)=a_(i), where a_(j)∈CT^(i). After that, node i sends apublication request to its successor towards its anchor controller a_(i)with the tuple {nid^(i) _(i), sn^(i) _(i), d^(i) _(i), LCL^(i) _(i)}.Each node v in the route from node i to controller a_(i) forwards thepublication request towards a_(i) and caches the tuple {nid^(v) _(i),sn^(v) _(i), d^(v) _(i), s^(v) _(i), LCL^(v) _(i)}. Once controllera_(i) receives the request, it stores the tuple {nid^(a i), sn^(a i)_(i), d^(a i) _(i), s^(a i) _(i), LCL^(a i) _(i)}. Hence, each nodeprocessing a publication request learns the route to the node issuingthe request, and the anchor controller is able to obtain the mappingneeded to redirect nodes sending subscription requests to the localcontrollers of node i.

The forwarding of subscription requests is handled in much the same waydescribed above for the case of publication requests. When node o hasdata for destination j∉CT^(o), it computes hash (j)=a_(j), wherea_(j)∈CT^(o) and sends its subscription request towards a_(j). Thesubscription request from node o regarding destination j states theidentifier of node j, its anchor controller a_(j), and LCL^(o). Whena_(j) receives o's request, it responds with the tuple {ni^(a j) _(j),sn^(a j) _(j) LCL^(a j) _(j)} and sends the response to the nearestcontroller it finds in LCL^(o).

Node o stores the tuple {nid^(o) _(j), sn^(o) _(j), LCL^(o) _(j)} inRT^(o) upon receiving the reply to its subscription. Data packets from oare then sent towards the controllers in LCL^(o) _(j) that are theclosest to node o. A data packet must specify the sender, thedestination, and the selected local controller of the destination. Thiscan be done by encapsulating the header of the packet stating the originand the destination with a header stating the origin and the selectedlocal controller of the destination. Once the packet reaches a relaynode y with an active route for the destination, the packet is forwardeddirectly to the destination itself, as long as the distance from node yto the destination is at most r hops.

In one embodiment of APDV, the functional steps required for itsoperation can be organized as illustrated in the pseudo-code listed inAlgorithm 4 below. That pseudo-code can be implemented in a variety ofways as part of a routing protocol designed for wired networks orwireless networks.

Algorithm 4 is represented as pseudo code for the overall functionalsteps of APDV:

APDVInit(i) { nid^(i) _(i) = i; cs^(i) _(i) = false; k^(i) _(i) = 0;sn^(i) _(i) = 1; CT^(i)=φ; NCT^(i)=φ; NCT^(i)=φ; NT^(i)=φ; LRT^(i)=φ;NLRT^(i)=φ; RT^(i)=φ; NRT^(i)=φ; LCL^(i)=φ; APDVSetTimer (i, SendHello,DEFAULT_HELLO_INERVAL); APDVSetTimer (i, CheckRoutingInfo,DEFAULT_UPDATE_INERVAL); APDVSetEventHandler (i, APDVHandleEvent);APDVSetSignalingPacketHandler (i, APDVHandleSignalingPacket); }APDVHandleEvent (i, eventType) { switch (eventType) { case SendHello: {SendHello(i); APDVSetTimer (i, SendHello, DEFAULT_HELLO_INERVAL); } caseCheckRoutingInfo: { CheckUpdate(i); APDVSetTimer(i, CheckRoutingInfo,DEFAULT_UPDATE_INERVAL); } } } HandleHello(i, packet) { UpdateNT(i,packet) UpdateCT(i, packet, i.Call (UCR)); i.Call(CAR); i.Call(CDR);UpdateLCL (i, i.Call(UCR)); SendLocalControllerUpdate(i); }APDVHandleSignalingPacket(i, packet, packetType) { Switch(packetType) {case HELLO { HandleHello(i, packet); } case AnchorUpdate; {HandleAnchorUpdate(i, packet); } } } APDV(i) { APDCInit(i) }

It should also be understood that while the figures and the abovedescription illustrate the present invention, they are exemplary only.They are not intended to be exhaustive or to limit the invention to theprecise forms disclosed, and obviously many modifications and variationsare possible in light of the above teaching. Others who are skilled inthe applicable arts will recognize numerous modifications andadaptations of the illustrated embodiments that remain within theprinciples of the present invention. Therefore, the present invention isto be limited only by the appended claims.

What is claimed is:
 1. A method of routing, the method comprising: (i)establishing a computer network comprising a plurality of nodes, eachnode of said plurality of nodes having a unique node identifier, theplurality of nodes including an anchor controller storing a mappinghaving addresses of each of the plurality of nodes; (ii) dynamicallyselecting a subset of said plurality of nodes to serve as controllers,said selecting based on a distributed algorithm that selects controllerssuch that each non-controller node is within a maximum distance from aminimum number of local controllers and wherein dynamically selectingcontrollers is performed in accord with predetermined ControllerAddition Rules, thereby dividing said plurality of nodes into aplurality of controller nodes and a plurality of destination nodes;(iii) maintaining routes to nearby destination nodes of each controllernode, thereby forming local controllers; (iv) maintaining routes to allcontrollers in each controller based on distance vectors; and (v) usingpublish-subscribe mechanisms in which destination nodes inform localcontrollers of routes to them and sources obtain routes to localcontrollers of the destination nodes, wherein the sources obtainingroutes includes: (a) sending a subscription request from a source nodeto the anchor controller, the subscription request including adestination node and a source node controller selected from the localcontrollers; (b) sending the mapping to the source node controller fromthe anchor controller; (c) forwarding the mapping to the source node;(d) sending a data packet based on the mapping to a destination nodecontroller of the destination node, the destination node being selectedfrom the local controllers; and (e) forwarding the data packet from thedestination node controller to the destination node.
 2. The method ofrouting of claim 1, wherein dynamically selecting controllers isperformed by nodes of said plurality of nodes deselecting themselves ascontrollers and wherein deselecting is performed in accord withController Deletion Rules.
 3. The method of routing of claim 1, whereinmaintaining routes to all controllers uses sequence numbers to restrictselection of next hops towards a given controller such that onlyneighbors with shorter distances to the controller can be considered assuccessors.
 4. The method of routing of claim 3, wherein maintainingroutes to all controllers include deleting controllers based onController Deletion Rules, and wherein controllers are deleted based onloss of connectivity to neighbors.
 5. The method of routing of claim 4,wherein controllers are deleted based on HELLOs from neighbors.
 6. Themethod of routing of claim 5, wherein controllers are updated based onan Update Controller rule.
 7. The method of routing of claim 1, whereinmaintaining routes to each nearby destination node of each controllernode in each controller is performed by: (i) each non-controller node dsending a publish message to each of its local controllers with amapping (d, {I¹ _(d) . . . I^(k) _(d)}), where I^(i) _(d) (1≤i≤k) is alocal controller for non-controller node d; (ii) each local controllerI^(i) _(d) of non-controller node d and each destination node betweensaid each local controller I^(i) _(d) and said non-controller node dreceiving said publish message stores a tuple stating an address ofnon-controller d, a next hop to non-controller d, and {I¹ _(d), . . . ,I^(k) _(d)}; and (iii) each non-controller node d selecting an anchornode a_(d) using a hash function; (iv) each non-controller node d sendssaid selected anchor node a_(d) the mapping (d, {I¹ _(d), . . . I^(k)_(d)}); and (v) said selected anchor node a_(d) and relay nodes betweend and a_(d) cache the mapping (d, {I¹ _(d), . . . , I^(k) _(d)}).
 8. Themethod of routing of claim 7, further including determining a path froma source node s to a destination node d by performing: (i) having a nodes identifying anchor a_(d) using said hash function; (ii) said node ssending said identified anchor a_(d) addresses d and (s, {I¹ _(s), . . ., I^(k) _(s)}) where I^(i) _(s) (1≤i≤k) is a local controller for nodes; (iii) anchor a_(d) returns a mapping (d, r) to node s by sending aresponse towards a local controller of node s; (iv) said localcontroller of node s passes said mapping (d, r) to node s, and (v) nodes sends data packets to d by sending them towards controller r.
 9. Themethod of routing of claim 1, wherein each non-controller node imaintains the following: a controller table (CT^(i)) that statesinformation about network controllers; a neighbor controller table(NCT^(i)) stating information reported by each neighbor ofnon-controller node i regarding all controllers elected in the network;a neighbor table (NT^(i)) stating information about all one-hop andtwo-hop neighbors of non-controller node i; a neighbor local routingtable (NLRT^(i)) stating routing information reported by each neighborregarding all destinations within two hops and some destinations withinr hops; a local routing table (LRT^(i)) stating routing informationabout all destinations within two hops and some destinations within rhops; a neighbor routing table (N RT^(i)) stating information reportedby each neighbor regarding distant destinations; and a routing table(RT^(i)) stating information about distant destinations.
 10. The methodof routing of claim 9, wherein entries in RT^(i) and N RT^(i) arepopulated by publish-subscribe signaling and wherein usingpublish-subscribe mechanisms uses a network-wide consistent hashfunction.
 11. The method of routing of claim 10, wherein publishing isperformed by: a node i using the network-wide consistent hash functionon a set of identifiers in CT^(i) to obtain hash(i)=a_(i), wherea_(i)∈CT^(i); node i sending a publication request towards its anchorcontroller a_(i) with a tuple {nid^(i) sn^(i) _(i), d^(i) _(i), LCL^(i)_(i),}; nodes in the route from node i to controller a_(i) forwardingthe publication request towards a_(i); and a_(i) caching a tuple{nid^(v) _(i) sn^(v) _(i), d^(v) _(i), s^(v) _(i), LCL^(v) _(i)}. 11.The method of routing of claim 10, wherein publishing is performed by: anode o using the network-wide consistent hash function on a set ofidentifiers in CT^(o) to obtain hash(i)=a_(j), where a_(j)∈CT^(o); nodeo sending a subscription request towards its anchor controller a_(j)stating an identifier of node j, its anchor controller a_(j), andLCL^(o); nodes in the route from node o to controller a_(j) forwarding apublication request towards a_(j); and a_(j) receiving o's request andsending a tuple {ni^(a j) _(j), sn^(a j) _(j) LCL^(a j) _(j)} to anearest controller it finds in LCL^(o); the nearest controller inLCL^(o) sending a tuple {ni^(1 j) _(j), sn^(1 j) _(j) LCL^(1 j) _(j)} too.
 12. The method of routing of claim 11, wherein node o stores a tuple{nid^(o) _(j), sn^(o) _(j), LCL^(o) _(j)} in RT^(jO) upon receiving areply to its subscription; and wherein node o sends data packets towardscontrollers in LCL^(o) _(j) that are closest to node o for transmissionto node i.
 13. The method of routing of claim 7, wherein maintainingroutes to named services or content objects is performed by performing:(i) each non-controller node d selecting an anchor node a_(o) using ahash function on a name of a service or content o that it wants topublish; (ii) each non-controller node d sends said selected anchor nodea_(o) a mapping (o, d, {l¹ _(d), . . . , l^(k) _(d)}); and (iii) saidselected anchor node a_(o) and the relay nodes between d and a_(o) cachethe mapping (o, d, {l¹ _(d), . . . , l^(k) _(d)}).
 14. The method ofrouting of claim 13, further including determining a path from asubscriber node s to a named service or content object o by performing:(i) a node s identifying anchor a_(o) using a hash function on the nameof the service or content object o required by s; (ii) said node ssending said identified anchor a_(o) the name o and (s, {l¹ _(s), . . ., l^(k) _(s)}), where l^(i) _(s) (1≤i≤k) is a local controller for nodes; (iii) anchor a_(o) returns a mapping (o, d, {l¹ _(d), . . . l^(k)_(d)}) to node s by sending a response towards a local controller ofnode s; (iv) said local controller of node s passes said mapping (o, d,{l¹ _(d), . . . , l^(k) _(d)}) to node s, and (v) node s sends requestfor object o to d by sending them towards a local controller of d. 15.The method of routing of claim 13, further comprising constructing aname of a service or content object to include a prefix component and asuffix component, with the prefix component being used in the hashfunction used to select an anchor controller for a named service orcontent object.
 16. The method of routing of claim 1, whereinestablishing a computer network establishes an ad hoc network.
 17. Themethod of routing of claim 16, wherein the ad hoc network is a wirelessnetwork.
 18. The method of routing of claim 14, wherein establishing acomputer network establishes an ad hoc network.
 19. The method ofrouting of claim 18, wherein the ad hoc network is a wireless network.